Backups, Encryption and Security

Backing up your files

I use 5 types of backups:

  • Home files online disaster recovery backup - runs couple times a week.
  • Local System restore for troubleshooting or disaster recovery - runs automatically once a week.
  • Cloud drive not really a backup but a convenient way to store and access working files and never lose them.
  • Settings sync: Basically dot files which are already backed up in Home but this is really convenience to allow easy editing and sync of settings between PCs.
  • Disk image: This takes an entire image of a disk to be copied directly to another disk. Used only occasionally for backing up bootable SD cards, servers or upgrading SSD/HDD.

Home files Incremental Backup (using Duplicati + your cloud provider)

I currently use Duplicati + Wasabi cloud storage ⭐. The latest version 2.0 is very easy to set up and use.

Home files Incremental Backup (using iDrive, paid)

Due to the time required to retrieve data, I consider this only for disaster recovery on PC failure. For working cloud storage of working files see the Cloud Drive section below.

iDrive is an economical large cloud backup (5TB) with great performance, from a company that has been around for a long time. The SW GUI is a bit dated though it works well through command line scripts on Linux, connected to an online cloud interface. Through the online interface you can monitor and control backup settings and set up automatic scheduled incremental backups of local and USB drives for any number of PCs. Update: Although very cheap, lately I encountered some bugs with high CPU usage of their scripts and no longer recommend it. I switched to Duplicati + wasabi cloud, as per above.

Home files Incremental Backup (using Duplicacy + your cloud provider)

Due to the time required to retrieve data, I consider this only for disaster recovery on PC failure. For working cloud storage of working files see the Cloud Drive section below.

Duplicacy CLI (User HOME files to Google Drive etc incremental encrypted online backup with version history).

  • Install: Using Go, by running: go get -u github.com/gilbertchen/duplicacy/...
  • Initialise backup by running within the folder to back up for example google drive:
    • duplicacy init -e vijay-nuc8-hades_home gcd://nuc8-hades/home/
    • To save authentication, put token in ~/.duplicacy/ and run: duplicacy set -key gcd_token -value .duplicacy/gcd-token.json
  • Quick usage once initially set up:
    • duplicacy backup (Run the backup for current folder)
    • duplicacy prune -keep 0:180 (Delete previous versions of backup older than 180 days)
    • duplicacy list (list revisions and dates of backups)
    • duplicacy restore -r 4 (rollback local files to revision 4)
    • duplicacy restore -r 7 "Documents/Sampledata.xlsx" (rollback/restore a specific file from revision 7)
  • You can also restore specific files only using the .duplicacy/filters file.
  • Download/restore all files in a revision into folder downloadall: mkdir -p downloadall cd downloadall duplicacy init vijay-dell-5290_home gcd://vijay-dell-5290/home/ (init local repo) duplicacy list (figure out which revision you want) duplicacy restore -r 10 (restore revision 10)
  • Reference:
  • More guides:

System Restore

Timeshift (System files backup to local disk)

  • Easy GUI tool for local “system restore” type functionality only, uses rsync for backend
  • Incremental snapshots of system files, including GRUB bootloader
  • User files are NOT included (/home and /root)
  • Need decent local disk space, or external HDD or NAS.
  • Suggest keeping last 2 weekly snapshots, and 1 monthly snapshot

Cloud Drive

Gnome and KDE Dolphin built-in Google Drive integration This works exceptionally well for a cloud drive integration very tightly in the OS. Better than Googles own tools (which arent even available for Linux) which are still buggy and a massive resource hog.

  • For Gnome you just need to add your Google account (under Online Accounts in settings) and your drive automotically shows up in the file browser (mounts as a network drive after clicking on it first time), along with your calender.
  • For KDE you just need the kio-gdrive add on and login under Accounts in system settings. It works slightly differently as it treats Google docs files as Open Office docs. In practice this always work fine for me so far for viewing and quick edits, I can edit a document in Open Office and save it (it convert back to Google doc on save), but a word of caution in case i messes with your google docs, or you make a lot of changes and it doesnt save! For heavy editing I always open it in Google docs.

Settings Sync

The basic idea is to symlink all of your config files into one folder (such as ~/.dotfiles/) and sync that to an online storage or git repo, and easily sync between many PCs.

You can do it manually or there are several options to automate this:

  • DotBot ⭐ which requires manual configuration (yaml file) but gives greater control and the option to add any application. I prefer this due to its flexibility. I personally use this on all my PCs and server

  • Mackup brew install mackup which is user-friendly and will autodetect configs to backup but is very simple and only supports specific apps.

Other potential tools

  • Simple local rsync based file-copy snapshots. For example I use this to back up my mobile USB drive, simply plug it into my home NAS then rsync the drive to the NAS RAID drive which is automatically backed up to cloud every night via Duplicati.
  • Free File Sync - Great for local backups, a GUI alternative to rsync.
  • GoodSync CLI - (GUI version is Windows/Mac only) Paid. Similar to rsync but has connectivity to many cloud providers.
  • Back In Time

Backup Tools that are not good or too buggy

  • ODrive
  • Backups using the built in Gnome backup tool (Deja Dup)
  • inSync (Slow, high CPU usage, not good with lots of files)

What to backup:

  • Users Home folder (or entire /home for all users), EXCLUDING:
    • -.cache/
    • -.logs/
    • -.wine/
    • -.wine32/
    • -.PlayOnLinux/
    • -.minikube/
    • -.steam/
    • -.config/google-chrome/
    • -.gradle/
    • -.dbus/
    • -.debug/
    • -.gvfs/
    • -.local/share/gvfs-metadata/
    • -.local/share/Trash/
    • -go/bin/
    • -go/pkg/
    • -Downloads/
    • -Insync/
    • -TEMP_not_backed/
  • Possibly manually installed system-wide programs:
    • /opt/
  • Other potential system configs and created files:
    • /etc
    • /root
    • /srv
    • /usr (/usr/local only, maybe /usr/share but nothing else)
    • /var (except /var/run, /var/cache, /var/tmp)
  • https://askubuntu.com/questions/222326/which-folders-to-include-in-backup

Disk Image Whole PC backup (partitions restore)

Options:

  • dd if=/dev/sda of=/dev/sdb bs=32M

  • Clonezilla Live (or DRBL live CD recommended)

Save and restore all apps on new PC

First copy configurations (dotfiles) from the old PC Home directory to the new one.

Then create a list of all installed packages. Use apt-clone (sudo apt-get install apt-clone). ALL packages will be included in the list including all sources. Some apps which were installed through other means (not a source url) will be listed in the console. Unclear if snap packages are included.

On old PC:

sudo apt-clone clone foo #This creates foo.apt-clone.tar.gz

On new PC:

sudo apt-clone restore foo.apt-clone.tar.gz

Another option is Aptik (originally FOSS, now paid)

Bitlocker and disk Encryption

  • USB drives:
  • Linux OS drive:
    • LUKS is default method for “full” encryption which has better performance and security. NOTE: It can ONLY be done while freshly installing Linux. You can also unlock your drive by inserting a USB drive with a key on it.
    • eCryptfs is another option which can be used to only encrypt the user’s home directory (and conveniently auto-decrypt when logging in using password). This can be done at any time after install. There is a bigger performance hit vs LUKS.
    • Could try Veracrypt but its more complicated.
  • Bitlocker can still be used for Windows partition on dual-boot systems. You may need to wrestle with Secure Boot bios setting if you rely on a TPM. However it is not possible to easily read Bitlocker drives in Linux.
  • Encrypt a local file/folder using PGP:
    • Make sure you have at least one PGP key on the PC (seahorse)
    • sudo apt-get install seahorse-nautilus
    • Right click on file file/folder and click Encrypt...

Create an encrypted USB Drive

Using Veracrypt (cross platform):

Veracrypt is very easy and cross platform but it has slower performance than Linux native solutions like LUKS.

Using LUKS (Linux only):

https://geekyshacklebolt.wordpress.com/2019/03/06/how-to-encrypt-usb-drives-with-luks/

LUKS is designed for Linux and not very cross platform. The great thing about LUKS is that it is fast and supports multiple passwords and keyfiles to unlock a single drive. Use sudo cryptsetup to open (i.e. decrypt) an encrypted volume, create encrypted partitions, modify settings and add/view decryption methods for a device.

Automount encrypted drive (on a headless server)

If you want to auto decrypt the drive on boot, try using crypttab with a keyfile, (similar to using fstab for auto-mounting non-encrypted volumes on boot). Technically it only decrypts drives on boot and not when USB inserted while running, see commands below to remount. A potential avenue to automount on insertion would be to use Hook Script with usbmount which detects the inserted device and runs a decrypt/remount script.

Add to /etc/crypttab:

tdrive UUID=e3a643ce-68c2-1dd2-521d-825400ba4872 /root/keyfile luks

(Get the UUID for the drive by checking symlinks in /dev/disk/by-uuid/) This will auto open/decrypt the volume based on the supplied keyfile and map it to /dev/mapper/tdrive. It is the same as running sudo cryptsetup luksOpen.

Add to /etc/fstab:

/dev/mapper/tdrive /media/tdrive ext4 defaults 0 0

When you reboot with the drive plugged in, it will automatically decrypt and mount by parsing crypttab then fstab. To remount by manually parsing both tab files at any time:

sudo cryptdisks_start tdrive
sudo mount -a

If you are looking for a GUI and/or CLI to easy manage mounting and encrypted mounts of any format (including Veracrypt, LUKS and even Bitlocker), try ZuluCrypt

Password saving and secrets

Use BitWarden⭐.

SSH Keys for logging in to services or servers

Many services such as Git repos, FTP, SSH remote terminal, and IOT devices, it is recommended to use SSH keys to authenticate instead of passwords. It’s certainly a better solution than saving passwords as cleartext files (default for some Git implementations).

  • The best practice is to have per client machine SSH keys that can easily be revoked by logging into the server with a strong password if required.
  • Each SSH key comes in a pair: private key and public key.
  • They are usually installed in the ~/.ssh directory.
  • The private key is secret and kept on the client.
  • The public key can be sent to the server you want to authenticate with (i.e. log into).
  • For web apps (e.g. Github, Bitbucket) simply log into the site, go to account settings and add your ssh public key.
  • For SSH login to a headless server, either run on client ssh-copy-id {username}@{remote_host} or append your public key to ~/.ssh/authorized_keys file.
  • Once set up, you no longer need a password to connect.
  • For Git, you need to switch to SSH style logins (change all your remote URLs in your local repos .git/config to SSH style URL)
  • SSH key pairs can be generated on the client with various algorithms (RSA, ED, EC) and bit length (the more the better, 2048 or more is great).
  • You don’t necessarily need to back up the private keys, they are seen as disposable and revocable, like a browser cookie or login token. Though they can be included in a secure regular backup of the users home folder, to avoid redoing it after erasing a PC.
  • The “Passwords and Keys” app built into Ubuntu can be used to generate and manage both keys and passwords. You can also generate a key pair using:
    ssh-keygen -t rsa -b 4096
  • You can optionally protect your private keys with a passphrase. That would mean they would need to be “unlocked” with the passphrase every time they need to be used (unless you are running SSH agent software that stores the decrypted key). If you store the private key only on an encrypted drive and it never moves, you are ok without a passphrase.

Passwordless sudo using U2F (e.g. Yubikey)

Set up Yubikey for Passwordless Sudo Authentication - DEV Community 👩‍💻👨‍💻

Passwordless Ubuntu Login using U2F

Passwordless login with Yubikey 5 NFC - Ask Ubuntu

Basically this creates the file common-u2f file which has the auth sufficient .. line (this can be changed to auth required .. if you want to use U2F as a second factor instead), then looks for the line @include common-auth in all the files under /etc/pam.d folder and adds @include common-u2f before that to enable passwordless auth. Note there may still be some apps in Ubuntu which are missed, for example the GNOME keyring, so you might still need the password for those. Or you could try this workaround for GNOME keyring.

To uninstall passwordless login, just search and remove the @include common-u2f from files under /etc/pam.d.